Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UCL Scotch support for SWIFT #42

Open
wants to merge 69 commits into
base: master
Choose a base branch
from
Open

UCL Scotch support for SWIFT #42

wants to merge 69 commits into from

Conversation

themkots
Copy link

No description provided.

Dan Giles and others added 30 commits March 15, 2023 10:41
DanGiles and others added 26 commits August 21, 2023 13:09
docs: New documentation on the installation and running with SWIFT
fix: Formatting according to format.sh (clang-format)
fix: Clean up of excess files
fix: Adding defs of idx_t and IDX_MAX
* Fix for infinite loop in Brent's method

* Fixed issues with missing unlocks/dependencies/debug checks for top level cells which aren't also super cells

* GEAR-RT: theory

* Deadlock detector

* Update spin/jet AGN model

* corrected number of items to write in tracers_io.h

* Fix new time-step for spin-jet scheme

* Grackle Cooling updates

* Make the EAGLE_ICs example load the PS2020 cooling tables and not the Wiersma+09 ones

* Update EAGLE_ICs README to mention how to load the Wiersma tables if needed

* Add a new potential MWPotential2014

* Fix jet initial velocities

* Fix EAGLE tracers' list of outputs

* Add forcing terms to the hydro equations

* Add barotropic EoS

* Add a function so that we can mark memory as deliberately leaked at the end of the program.

* Update README-Scotch.md

* Update README-Scotch.md

Co-authored-by: Ilektra Christidi <[email protected]>

* Update README-Scotch.md

Co-authored-by: Ilektra Christidi <[email protected]>

* Update README-Scotch.md

Co-authored-by: Ilektra Christidi <[email protected]>

* Update README-Scotch.md

Co-authored-by: Ilektra Christidi <[email protected]>

---------

Co-authored-by: Yolan Uyttenhove <[email protected]>
Co-authored-by: Matthieu Schaller <[email protected]>
Co-authored-by: Mladen Ivkovic <[email protected]>
Co-authored-by: Filip Husko <[email protected]>
Co-authored-by: Yves Revaz <[email protected]>
Co-authored-by: Darwin <[email protected]>
Co-authored-by: Ilektra Christidi <[email protected]>
* fix: Passing architecture file as flag

* Fixed prevailing SCOTCH invocation with configure script; made ArchData globally visible.

* Fixed picking scotch when also have metis/parmetis

* Fixed picking scotch when also have metis/parmetis

* Fixed populating partition struct with scotch target arch file before calling initial partitioning.

* Prevent destruction and empty regeneration of archdat structure.

* Edited swift_fof.c to match target file updates of swift.c

* Minor edits: replaced a HAVE_PARMETIS with a HAVE_METIS, edit a message to show either SCOTCH or METIS used, changed strategy flag from SCOTCH_STRATQUALITY to SCOTCH_STRATBALANCE

* Added a mesage to notify when a successful SCOTCH repartition takes place

* Dropping reference to METIS in pick_scotch

* Changed format specifiers for SCOTCH weights reporting in debug check - now acceptable for both 32 and 64 bit versions.

* Merged two separate ifdef chunks of HAVE_SCOTCH for better visibility

* Dressed up naked mallocs; averaged [div by 2.0] the edge loads; removed commented out and unused code.

* Updated string for averaged and not summed weights

---------

Co-authored-by: Dan Giles <[email protected]>
Co-authored-by: Themis Kotsialos <[email protected]>
* Initialised return_edge variable

* Removed a typedef for idx_t and a define for SCOTCH_Idx;

* Removed typedef from metis/parmetis - only code chunk.
…83647); makes scotch64 and scotch-local balance slightly better.
…33)

* Update partition.c

* Update configure.ac

* Removal of refine in Scotch
* Update debug.c - minor edits

* Update engine.c

* Update swift.c

* Update swift_fof.c

* Update partition.c

* Update partition.c
* Update README-Scotch.md

* Update README-Scotch.md - Warning on compiler inclusion preferences

* Update README-Scotch.md - prod runs table updates 01

* Update README-Scotch.md table runs 02

* Update README-Scotch.md remove old table

* Update README-Scotch.md added note for Rome and Milan type of Cosma 8 nodes

* Update README-Scotch.md

* Update README-Scotch.md - prod runs table 03

* Update README-Scotch.md - large simulations dump minimisation

---------

Co-authored-by: themkots <[email protected]>
@MatthieuSchaller
Copy link
Member

@pwdraper, for the records, could we list here the issues we had not resolved?
I think the main one was the lack of 64-bits support and the inability for scotch to evolve a previous partition?

@themkots
Copy link
Author

themkots commented Jul 3, 2024

@MatthieuSchaller , @pwdraper , @DanGiles
There are the things that @MatthieuSchaller mentions just above
(32-bit linking where we expected 64-bit one, no refinement -- for the scotch-version, not the ptscotch one --
we had no time investigating ptsccotch, as per this issue messages, and I find hard to believe that ptscotch does not do
an incremental refinement -- otherwise how can it do parallelisation/mpi stuff ?? --)
There is also the hard-wired graph-connectivity issue that metis/parmetis seem not to cross over, but scotch does, i.e. when it finds more that <hard wired number? cannot remember right now - is in src - 64? something else?> nodes able to communicate with one another, it tries to leverage this large connectivity for load/compute/comms balance, but this results in runtime error from code when discovering so in graph partitioning call.

Wish we had more time to delve into those issues, and a) have 64-bit clean builds, b) explore the obvious strategy of incremental refinement, c) explore large connectivity for better / lower than say 5% gaps.

Ta.

@MatthieuSchaller
Copy link
Member

Thanks Themys!

For the 64-bits build, do you recall whether it's a problem of creating a clean build or what is a more fundamental lack of support? My memory says it's the latter.

@themkots
Copy link
Author

themkots commented Jul 3, 2024

Yes, it is their source that links the 32 bit one --- could attempt a direct edit, or send a PR, if in github somehow, but not right now...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants